feat(hive): Restore the Thrift methods removed by HIVE-26537 for 4.2.0 - #1636
Merged
Merged
Conversation
lfrancke
force-pushed
the
feat/hive-restore-thrift-methods
branch
from
September 14, 2026 11:16
b2aed36 to
2ac61fe
Compare
lfrancke
marked this pull request as ready for review
September 14, 2026 11:19
lfrancke
moved this from Development: Waiting for Review
to Development: In Progress
in Stackable Engineering
This was referenced Sep 14, 2026
Hive 4.0.1 removed get_table and get_table_objects_by_name from the metastore Thrift interface. Any client built against a Hive 2.3 metastore client still calls them, and that is what Spark bundles: 2.3.9 in Spark 3.5, 2.3.10 in Spark 4.0 through 4.2. Iceberg's HiveCatalog reaches the metastore through IMetaStoreClient and therefore uses whichever client Spark supplies, so an Iceberg job against a Hive 4.0.1 or newer metastore fails. This restores the methods as they were in Hive 4.0.0.
lfrancke
force-pushed
the
feat/hive-restore-thrift-methods
branch
from
September 14, 2026 11:47
2ac61fe to
339a2e8
Compare
lfrancke
added this pull request to stack #1638
September 14, 2026 12:23
lfrancke
moved this from Development: In Progress
to Development: Waiting for Review
in Stackable Engineering
NickLarsenNZ
moved this from Development: Waiting for Review
to Development: In Review
in Stackable Engineering
NickLarsenNZ
self-requested a review
September 18, 2026 06:19
NickLarsenNZ
previously approved these changes
Sep 18, 2026
#1638, #1641 and #1664 landed on main and appended to the same Fixed section and link-definition block this branch adds to. Merged rather than rebased so the reviewed commit stays untouched and the PR gains a commit rather than a rewrite. The resolution is mechanical: both sides' entries kept, link definitions sorted.
Member
Author
|
Had to fix the changelog conflict. |
NickLarsenNZ
moved this from Development: In Review
to Development: Done
in Stackable Engineering
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Hive 4.0.1 removed get_table and get_table_objects_by_name from the metastore Thrift interface. Any client built against a Hive 2.3 metastore client still calls them, and that is what Spark bundles: 2.3.9 in Spark 3.5, 2.3.10 in Spark 4.0 through 4.2. Iceberg's HiveCatalog reaches the metastore through IMetaStoreClient and therefore uses whichever client Spark supplies, so an Iceberg job against a Hive 4.0.1 or newer metastore fails.
This restores the methods as they were in Hive 4.0.0.
This also fixes a related problem: We set the version to build to include the
-stackablepostfix. And we'd expect Maven to use those artifacts it builds in the same build process. Well...it didn't because Hive has - in some places - hardcoded the wrong version which means that Hive would not actually use the things we build but instead downloads the dependencies from Maven Central. This fixes that as well.I ran the Hive kuttl test against this and they all succeeded. That doesn't say much because they don't test the Iceberg behavior which is why there is another PR for hive-operator upcoming that adds a test there.
Definition of Done Checklist
Note
Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.
Please make sure all these things are done and tick the boxes
TIP: Running integration tests with a new product image
The image can be built and uploaded to the kind cluster with the following commands:
See the output of
boilto retrieve the image manifest URI for<MANIFEST_URI>.